Fix leaks, round-trip failures and config bugs from the second bug hunt - #90

Second bug hunt. It used new fuzzers (numeric entities, JSON, brackets and slashes, verb-led lines, --re-mask chains) plus audits of the configuration layer and of the command-line and I/O layer. All entries are in CHANGELOG.md under [Unreleased].

Masking leaks

• Masks of military IDs, BR numbers, passports, tax numbers and units could equal the original, which left the number open. About 1 military ID in 100 was affected (only two digits of six change), and 1 №5дск in 10. These masks are now regenerated until they differ from the original and from the open text. Masks that already differed are unchanged.
• Names in brackets, slashes or asterisks: (Петренко Іван Петрович) kept the surname open and lost the closing bracket.
• Видати Петренку Івану: the infinitive was masked as a surname.
• always_mask and custom-pattern masks were masked a second time as surnames, so unmasking returned a mask instead of the original. A custom pattern could also match a protected phrase's placeholder, and the phrase disappeared.
• system.encoding: auto read UTF-16 as cp1251: nothing was masked, and the run still exited with 0. A BOM now decides the encoding, and round trips keep the file byte for byte.

Unmasking did not restore the original

• JSON: every string was unmasked with its own occurrence counters, while masking numbers occurrences across the whole document. Both sides now treat all strings of a file as one text (datamasking/_jsontext.py).
• An order-number mask could equal part of an open number (п. 1.2).
• *старший матрос …*: the rank was masked without its first word.
• Text dates lost the month's case, and 123 травня 2025 was masked starting from 23.
• JSON or text is now taken from the mapping (input_format), not from the masked file's extension.

Configuration

• exclusions match all apostrophe variants.
• The local template no longer erases the shared lists.
• max_name_length is null in the shared config.
• Unknown ENV booleans are skipped with a warning; before, DATA_MASKING_ENCRYPT_OUTPUT=y produced a plaintext mapping.
• ENV preserve_case now wins over the file.
• router_rules filters combine with enable_*.
• Fixed: password_generation: "false", fractional integers, --re-mask warn counts, empty sections.
• Tracebacks replaced with errors: hash_algorithm case, backup_suffix separators, non-string logging.level, non-string section keys.
• diagnose returns 1 for bad files.

Command line and files

• --only/--exclude also control other document numbers.
• JSON input with a BOM is accepted.
• unmask applies the same size limit as mask.
• --password works together with encrypt_output: true.
• A password that is not UTF-8 is rejected before any file is written.
• -o pointing to a hard link of the input is refused.
• Empty masks in a mapping are rejected.
• The report shows the passes actually made.
• Tracebacks replaced with errors: damaged .enc v2 headers, --init-config into a missing folder.
• unmask exits with 2 for wrong usage. Two assertions in tests/test_to_version.py change from 1 to 2.

Tests

New test files. The share failing on main is shown for each:

• Full suite: 1732 passed, 1 skipped. mypy and flake8 are clean.
• Fuzzing after the fixes, about 6000 documents: 0 round-trip failures and 0 failures in 1000 --re-mask chains.
• The only leaks the fuzzers still report are two-word names split across lines, a known limitation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XT6iUWaQgahXDB9TWX9Bq7